Shorthand on graphical keyboards (hereafter “shorthand-on-keyboard”) or Shorthand on a Keyboard as Graph (sokgraph), represent an input method and system for efficiently entering text without a physical keyboard, typically using a stylus. Shorthand-on-keyboard enables the user to trace letter or functional keys on the graphical keyboard to enter words and commands into a computer. Experienced users partly or completely memorize the geometric patterns of frequently used words and commands on the keyboard layout and may draw these patterns based on memory recall using, for example, a digital pen.
Word-level recognition-based text entry systems such as shorthand-on-keyboard and handwriting/speech recognition as well as text prediction systems all rely on some form of lexicon that defines the set of words that these systems recognize. The input of the user is matched against choices in the lexicon. Words not included in the lexicon are usually not automatically recognized. In such a case, a special mode has to be provided. For example, in shorthand-on-keyboard the user may initially check a candidate list (N-best list). If no choice on the candidate list is the intended word, the user decides if the patterns drawn were incorrect. If the patterns drawn were correct, the user realizes the word intended is not in the lexicon. The user then enters the new word in the lexicon by tapping the individual letters. Ideally, the lexicon comprises all words a particular user needs to write, no more no less. A lexicon that is either too large or too small can introduce problems to the user.
A larger lexicon could present certain challenges, since it tends to reduce the recognition accuracy due to the likelihood of a greater number of distracters for each user input. In any language, there tends to be a core set of vocabulary that is common to all individuals. Beyond this core set, vocabulary tends to be specialized for a particular individual. For instance, an engineer may compose emails comprising highly technical terms and abbreviations for a particular field or business area. For other users, these specialized terms can be irrelevant and can introduce noise in the recognition process, making the recognition process less robust.
A smaller lexicon is typically a more robust lexicon in that user input is more likely to be correctly recognized, provided the intended word is in the lexicon. A smaller lexicon provides more flexibility and tolerance for the input of the user, allowing the input to be imprecise and inaccurate compared to the ideal form of the intended input choice. A further advantage of a small lexicon is that the search space is smaller. Consequently, a small lexicon allows reduction in the latency of a search. This is especially important in mobile devices where processing power is severely limited.
However, when a small lexicon does not contain the word the user needs, the user experience can be frustrating. A user does not know, prior to entry, whether a word is in the lexicon, causing uncertainty for the user. The lack of recognition of a word by a conventional system can occur either when the word is input incorrectly or when the word is not in the lexicon. Consequently, it can be difficult for the user to determine why a word is not recognized. In general, the user cannot know whether a word is in the lexicon except by repeatedly trying the word. When the user is certain that the word is not in the lexicon, the user adds that word to the lexicon via an interface provided by the recognition system by tapping as described earlier. A smaller lexicon requires a user to add words to the lexicon more often.
There are several conventional solutions to the lexicon size issue. A commonly used method is to use a large lexicon and then take advantage of higher order language regularities such as a word-level trigram-model to filter out highly unlikely candidates. The downside of a language model approach is generally the overhead of creating and making efficient use of a large language model. Moreover, a language model can introduce errors and mistakenly filter out the intended words. This is especially true if the language model is generic rather than customized to a particular user. In practice, efficient customization of a language model is difficult. Furthermore, a language model is difficult to integrate with a recognition technique that already has a high precision, such as shorthand-on-keyboard.
An alternative conventional approach creates a customized lexicon for a user by mining the written text generated by the user, for example, written emails and other documents. Although this approach does result in a lexicon more closely tailored to a specific user, a previously written corpus generated by a user may be to be too small to cover all of the desired words. Furthermore, in practice, it is difficult to write a computer program code that can open and read all and various email and document formats that the user may be using. This approach often requires the user to locate and select the previous written documents, which is inconvenient for the user. A customized lexicon may also be difficult to carry over across different devices.
Although these conventional solutions are adequate for their intended purpose, it is desirable to find a solution that enables a lexicon to have a relatively small number of irrelevant distracters to the user's desired input and yet allows easy access to almost all words the user may need, including more specialized words that are infrequently used by most users. Overall, there is a desire to include all words possibly needed by the user in a very large lexicon. However a very large lexicon implies that more words match the pattern drawn on the keyboard given the same matching threshold, reducing the signal-to-noise ratio in the input system. Consequently, a larger lexicon corresponds to less flexibility and robustness to the user. Thus, there is a need for a lexicon configuration for a shorthand-on-keyboard system that balances ease of use with flexibility and robustness.
Another challenge to a conventional shorthand-on-keyboard input method is a requirement of entering text exactly at the word level, one word at a time. Some words are long. For relatively new users, it can be cognitively difficult to draw a long word with shorthand-on-keyboard in one stroke. This difficulty is particularly acute in some European languages in which compound long words are more common than in English. Furthermore, a user can find entry more convenient if common affixes can be drawn as a separate stroke from the stem of the word. For example, to write the word “working” with shorthand-on-keyboard, the user may wish to draw the pattern of w-o-r-k on a graphical keyboard, then draw i-n-g and combine the two into one word. Thus, there is a need for an effective system and method to automatically combine partial words on the keyboard (“sokgraphs”) into one word as intended by the user.
What is therefore needed is a system, a computer program product, and an associated method for a system and method for improving text input in a shorthand-on-keyboard interface. The need for such a solution has heretofore remained unsatisfied.